Automated optical inspection (AOI) image classification method, system and computer-readable media

ABSTRACT

An automated optical inspection (AOI) image classification method includes sending a plurality of NG information of a plurality of samples from an AOI device into an Artificial Intelligence (AI) module; performing discrete output calculation on the NG information of the samples by the AI module to obtain a plurality of classification information of the samples; performing kernel function calculation on the classification information of the samples by the AI module to calculate respective similarity distances of the samples and performing weighting analysis; based on weighting analysis results of the samples, judging classification results of the samples; and based on the classification results of the samples, performing classification of the samples.

CROSS-REFERENCE TO RELATED ART

This application claims the benefit of Taiwan application Serial No.106137265, filed Oct. 27, 2017, the disclosure of which is incorporatedby reference herein in its entirety.

TECHNICAL FIELD

The disclosure relates in general to an automated optical inspection(AOI) image classification method, system and computer-readable media.

BACKGROUND

Automated optical inspection (AOI) image inspection systems have highspeed and high accuracy. AOI systems inspect by using machine visualtechnology and thus may replace manual inspection. In industrymanufacturing process, AOI systems may sense the surface status of thefinished product and then inspect the defects such as foreign matter orabnormal pattern by computer image processing algorithm.

However, when tradition AOI systems inspect defects, leakage (referringthe situation that the defects are not inspected and thus failed samplesare wrongly judged as good samples) or overkill (referring the situationthat good samples are wrongly judged as failed samples) may occur. Asfor leakage or overkill, in tradition, manual inspection is relied toimprove the accuracy rate.

Further, in tradition AOI technology, the sample (for example, thewafer) will be fed into training characteristic models, classificationmode, defects classification and the like procedures. Therefore, it isdifficult to achieve fast production change in tradition AOI technology.

SUMMARY

According to one embodiment, provided is an automated optical inspection(AOI) image classification method. The method includes: feeding aplurality of NG information of a plurality of samples from an AOI deviceinto an artificial intelligence (AI) training module; performingdiscrete output on the plurality of NG information of the samples by theAI training module to generate a plurality of classification informationof the samples; performing kernel function on the plurality ofclassification information of the samples by the AI training module tocalculate respective similarity distances of the samples and to performweighting analysis; performing classification determination based onweight analysis results of the samples, to determine respectiveclassification results of the samples; and based on the respectiveclassification results of the samples, classifying the samples.

According to another embodiment, provided is an automated opticalinspection (AOI) image classification system including: an AOI device,for performing automated optical inspection on a plurality of samples toobtain respective OK information or NG information of the samples; andan artificial intelligence (AI) training module coupled to the AOIdevice. The AI training module receives the plurality of NG informationof the samples from the AOI device. The AI training module performsdiscrete output on the NG information of the samples to generate aplurality of classification information of the samples. The AI trainingmodule performs kernel function on the plurality of classificationinformation of the samples to calculate respective similarity distancesof the samples and to perform weighting analysis. The AI training moduleperforms classification determination based on weight analysis resultsof the samples, to determine respective classification results of thesamples.

According to an alternative embodiment, provided is A computer-readablemedia, when loaded by an automated optical inspection (AOI) imageclassification system, the AOI image classification system executing theAOI image classification method as described above.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a functional block diagram of an automated opticalinspection (AOI) image classification system according to an embodimentof the application.

FIG. 2 shows a flow chart of an automated optical inspection imageclassification method according to an embodiment of the application.

FIG. 3A and FIG. 3B show weight characteristic analysis based on kernelfunction according to an embodiment of the application.

FIG. 4A (prior art) and FIG. 4B show classification results of the priorart and classification results of an automated optical inspection imageclassification system according to an embodiment of the application,respectively.

FIG. 5A and FIG. 5B show weight characteristic analysis based on kernelfunction according to an embodiment of the application.

In the following detailed description, for purposes of explanation,numerous specific details are set forth in order to provide a thoroughunderstanding of the disclosed embodiments. It will be apparent,however, that one or more embodiments may be practiced without thesespecific details. In other instances, well-known structures and devicesare schematically shown in order to simplify the drawing.

DESCRIPTION OF THE EMBODIMENTS

Technical terms of the disclosure are based on general definition in thetechnical field of the disclosure. If the disclosure describes orexplains one or some terms, definition of the terms is based on thedescription or explanation of the disclosure. Each of the disclosedembodiments has one or more technical features. In possibleimplementation, one skilled person in the art would selectivelyimplement part or all technical features of any embodiment of thedisclosure or selectively combine part or all technical features of theembodiments of the disclosure.

FIG. 1 shows a functional block diagram of an automated opticalinspection (AOI) image classification system according to an embodimentof the application. FIG. 2 shows a flow chart of an automated opticalinspection image classification method according to an embodiment of theapplication. FIG. 3A and FIG. 3B show weight characteristic analysisbased on kernel function according to an embodiment of the application.FIG. 4A and FIG. 4B show classification results of the prior art andclassification results of an automated optical inspection imageclassification system according to an embodiment of the application,respectively.

In FIG. 1, the AOI image classification system 100 includes an AOIdevice 120 and an AI (artificial intelligence) training module 150. Theclassification result from the AI training module 150 may be sent to aclassification device 170 for classification.

The AOI device 120 includes, for example but not limited by, AOI imageprocessing software, AOI sensing system, AOI inspection machine and soon. The structure of the AOI device 120 is not specified here.

The AI training module 150 is coupled to the AOI device 120 forperforming defect-overkill determination.

The classification device 170 is for sending the classified samples toclassification regions for subsequent sample manufacturing process,sample defect repair or sample disposal. The classification device 170includes, for example but not limited by, any combination of a pneumaticcylinder, a conveyor, a cantilever mechanism, a robot arm, and acarrier. The structure of the classification device 170 is not specifiedhere.

In an embodiment of the application, the AOI device 120 performs opticalinspection (step 210). The optical inspection result may indicate OK(success) inspection result and NG (failed) inspection result. OK(success) inspection result indicates that after inspection, the sampleis determined as good but leakage case may possibly exist. NG (failed)inspection result indicates that after inspection, the sample isdetermined as failed but overkill case may possibly exist.

Thus, in an embodiment of the application, whether the AI trainingmodule 150 is trained or not is determined (step 215). If the AItraining module 150 is not trained yet (that is, the AI training module150 is in initial training) or the AI training module 150 should bemanually adjusted, then the inspector manually inspects to determinewhether leakage or overkill exists (step 220).

In step 225, the samples enter into the AI training module 150, whereinthe samples may come from the AOI device 120 or from manual inspection.If the samples come from the AOI device 120, then the samples are whatare determined as NG samples by the AOI device 120. If the samples comefrom manual inspection, then the samples are what are determined as NGsamples by the AOI device 120 but determined as OK by manual inspection(i.e. the samples are overkilled by the AOI device 120). In anembodiment of the application, information of the NG samples is referredas NG information.

In step 230, the AI training module 150 performs discrete outputcalculation on the samples, for obtaining each classification result ofthe samples.

In an embodiment of the application, the AI training module 150 performskernel function on the classification result of the discrete outputcalculation (step 235). In kernel function, similarity measurement orsimilarity distance of the classification result of the samples iscalculated, and weighting analysis is performed on similarity distances.That is, based on similarity distance (or similarity measurement), thesample classification information having high similarity (or smallsimilarity distance) is assigned with high weighting while the sampleclassification information having low similarity (or large similaritydistance) is assigned with low weighting. The weight analysis result mayfurther be normalized. By such, classification result corresponding toblurred defects will be more distinguishable.

In step 240, based on kernel function calculation result, theclassification results are determined as one of classification 1−n (nbeing a natural number). The classification determination result in step240 may be fed into the AI training module 150 for training the AItraining module 150 to increase robustness of the AI training module150. By so, the classification result difference is more distinguishable(that is, the classification determination is easier).

Further, based on classification determination result in step 240, theclassification device 170 performs classification (step 245) and sendsthe classified samples to classification regions for subsequent samplemanufacturing process, sample defect repair or sample disposal and soon.

Refer to FIG. 3A which describes that classification results areimproved by kernel function according to an embodiment of theapplication. In FIG. 3A, the number of classification is n (n being anatural number). Assume that after discrete output, the classificationinformation [X1, X2, . . . , Xn] of the sample 310 is obtained.

The respective similarity measurement (or the similarity distance)between the classification information [X1, X2, . . . , Xn] of thesample 310 and each ideal classification “OK”, “ng1”, . . . , “ng(n−1)”is calculated. For example, the ideal classification information of theclassification “OK” is [1.0, 0.0, . . . , 0.0, 0.0, 0.0]; the idealclassification information of the classification “ng1” is [0.0, 1.0,0.0, . . . , 0.0, 0.0]; and the ideal classification information of theclassification “ng(n−1)” is [0.0, 0.0, 0.0, 0.0, . . . , 1.0].

In an embodiment of the application, the respective similaritymeasurement (or the similarity distance) between the classificationinformation of the sample and each ideal classification “OK”, “ng1”, . .. , “ng(n−1)” may be calculated by algorithms. Algorithms suitable in anembodiment of the application include, for example but not limited by,Euclidean Distance algorithm, Manhattan Distance algorithm, ChebyshevDistance algorithm, Minkowski Distance algorithm, Cosine algorithm,Mahalanobis Distance algorithm, Jaccard similarity coefficient algorithmor the like.

As shown in FIG. 3A, after similarity measurement, n similaritydistances Δd1=a1, Δd2=a2, . . . , Δdn=an are obtained, wherein “Δd1”indicates the similarity distance between the sample classificationinformation and the ideal classification information “OK”, “Δd2”indicates the similarity distance between the sample classificationinformation and the ideal classification information “ng1”, . . . and“Δdn” indicates the similarity distance between the sampleclassification information and the ideal classification information“ng(n−1)”.

The weight analysis is performed and accordingly, the classificationinformation having high similarity is assigned with high weight. In anembodiment of the application, an example of weight analysis is asfollows. All similarity distances are added (Σ₁ ^(n)ai=a1+ . . . an) andthe respective similarity distance is divided by the summation of thesimilarity distances to obtain the following values: “a1/(Σ₁ ^(n) ai)”,“a2/(Σ₁ ^(n) ai)”, . . . and “an/(Σ₁ ^(n) ai)”. Normalization isperformed (which is for explanation, not to limit the application) andaccordingly, summation of all classification possibility is forexample 1. In an embodiment, the following normalization equation isused:

$\frac{{\sum\limits_{1}^{n}{ai}} - {a\; 1}}{w \times {\sum\limits_{1}^{n}{ai}}},\ldots\mspace{14mu},\frac{{\sum\limits_{1}^{n}{ai}} - {an}}{w \times {\sum\limits_{1}^{n}{ai}}}$to obtain weighted classification information of the sample, wherein “w”is a normalization parameter.

The similarity distances (the similarity measurement) of the sample areassigned with different weighting. The small similarity distance (thesmall similarity measurement) is assigned with high weighting and thehigh similarity distance (the high similarity measurement) is assignedwith low weighting.

Thus, the normalized weight analysis result may be used to determine theclassification of the sample.

An example of how to improve the classification determination accuracyin an embodiment of the application is described. As shown in FIG. 3B,the embodiment of the application has improved classificationdetermination on the sample 310 by the kernel function. After discreteoutput calculation, the classification information of the sample 310 is[0.3, 0.3, 0.1, 0.2, 0.1].

The respective similarity measurement (similarity distance) between theclassification information [0.3, 0.3, 0.1, 0.2, 0.1] of the sample 310and each ideal classification “OK”, “ng1”, “ng2”, “ng3” and “ng4” is tobe calculated. For example but not limited by, ideal classificationinformation of the classification “OK” is [1.0, 0.0, 0.0, 0.0, 0.0];ideal classification information of the classification “ng1” is [0.0,1.0, 0.0, 0.0, 0.0]; ideal classification information of theclassification “ng2” is [0.0, 0.0, 1.0, 0.0, 0.0]; ideal classificationinformation of the classification “ng3” is [0.0, 0.0, 0.0, 1.0, 0.0];and ideal classification information of the classification “ng4” is[0.0, 0.0, 0.0, 0.0, 1.0].

In an embodiment of the application, the respective similaritymeasurement (similarity distance) between the classification informationof the sample 310 and each ideal classification “OK”, “ng1”, “ng2”,“ng3” and “ng4” is calculated by the algorithm.

As shown in FIG. 3B, in a possible embodiment of the application, aftersimilarity measurement calculation, the five similarity distance is Δd1(or a1)=10, Δd2 (or a2)=40, Δd3 (or a3)=120, Δd4 (or a4)=100, and Δd5(or a5)=80. Δd1 (or a1) refers the similarity distance between theclassification information of the sample 310 and the idealclassification information “OK”; Δd2 (or a2) refers the similaritydistance between the classification information of the sample 310 andthe ideal classification information “ng1”; Δd3 (or a3) refers thesimilarity distance between the classification information of the sample310 and the ideal classification information “ng2”; Δd4 (or a4) refersthe similarity distance between the classification information of thesample 310 and the ideal classification information “ng3”; and Δd5 (ora5) refers the similarity distance between the classificationinformation of the sample 310 and the ideal classification information“ng4”.

Weighting analysis is performed and accordingly the classificationinformation having high similarity is assigned by heavy weighting. In anembodiment of the application, an example of weighting analysis is asfollows. All similarity distances are added (Σ₁ ^(n) ai=a1+ . . . an)and the respective similarity distance is divided by the summation ofthe similarity distances to obtain the following values: “a1/(Σ₁ ^(n)ai)”=10/350, “a2/(Σ₁ ^(n) ai)”=40/350, “a3/(Σ₁ ^(n) ai)”=120/350,“a4/(Σ₁ ^(n) ai)”=100/350, and “a5/(Σ₁ ^(n) ai)”=80/350. Then, the aboveequation is used in the normalization calculation (w=n−1=5−1=4).

$\frac{{\sum\limits_{1}^{n}{ai}} - {a\; 1}}{w \times {\sum\limits_{1}^{n}{ai}}} = {{\left( {{\sum\limits_{1}^{n}{ai}} - {a\; 1}} \right)/\left\lbrack {w*\left( {{a\; 1} + \ldots + {a\; 5}} \right)} \right\rbrack} = {{\left( {350 - 10} \right)/\left( {4*350} \right)} = 0.2425}}$$\frac{{\sum\limits_{1}^{n}{ai}} - {a\; 2}}{w \times {\sum\limits_{1}^{n}{ai}}} = {{\left( {{\sum\limits_{1}^{n}{ai}} - {a\; 2}} \right)/\left\lbrack {w*\left( {{a\; 1} + \ldots + {a\; 5}} \right)} \right\rbrack} = {{\left( {350 - 40} \right)/\left( {4*350} \right)} = 0.2225}}$$\frac{{\sum\limits_{1}^{n}{ai}} - {a\; 3}}{w \times {\sum\limits_{1}^{n}{ai}}} = {{\left( {{\sum\limits_{1}^{n}{ai}} - {a\; 3}} \right)/\left\lbrack {w*\left( {{a\; 1} + \ldots + {a\; 5}} \right)} \right\rbrack} = {{\left( {350 - 120} \right)/\left( {4*350} \right)} = 0.165}}$$\frac{{\sum\limits_{1}^{n}{ai}} - {a\; 4}}{w \times {\sum\limits_{1}^{n}{ai}}} = {{\left( {{\sum\limits_{1}^{n}{ai}} - {a\; 4}} \right)/\left\lbrack {w*\left( {{a\; 1} + \ldots + {a\; 5}} \right)} \right\rbrack} = {{\left( {350 - 100} \right)/\left( {4*350} \right)} = 0.1775}}$$\frac{{\sum\limits_{1}^{n}{ai}} - {a\; 5}}{w \times {\sum\limits_{1}^{n}{ai}}} = {{\left( {{\sum\limits_{1}^{n}{ai}} - {a\; 5}} \right)/\left\lbrack {w*\left( {{a\; 1} + \ldots + {a\; 5}} \right)} \right\rbrack} = {{\left( {350 - 80} \right)/\left( {4*350} \right)} = 0.1925}}$

In other words, the similarity measurement (similarity distances) isassigned by different weighting. The small similarity measurement(similarity distance) is assigned by high weighting; and the largesimilarity measurement (similarity distance) is assigned by lowweighting.

Thus, from the normalization weighting analysis, the sample 310 shouldbe determined as “OK” classification, and the determination has lowoverkill possibility. On the contrary, if the classificationdetermination is based on the discrete output classification information[0.3, 0.3, 0.1, 0.2, 0.1], then the sample may be classified as theclassification “ng1”.

As shown in FIG. 4A, in prior art, the classification results of thesamples 410-450 may be obtained by discrete output calculation. Forexample, after discrete output calculation by the AI training module150, the classification information of the sample 410 is [0.1, 0.0, 0.8,0.0, 0.1]. Thus, it is good to classify the sample 410 as theclassification “ng2” because the possibility “0.8” of the classification“ng2” is far higher than the possibility of other four classifications.

Five classification “OK, ng1-ng4” of FIG. 4A and FIG. 4B are fordescription, not to limit the application.

Similarly, the classification information of the sample 420 is [0.0,0.0, 0.0, 0.1, 0.9]. Thus, it is good to classify the sample 420 as theclassification “ng4” because the possibility “0.9” of the classification“ng4” is far higher than the possibility of other four classifications.

The classification information of the sample 430 is [0.0, 0.0, 0.0, 1.0,0.0]. Thus, it is good to classify the sample 430 as the classification“ng3” because the possibility “1.0” of the classification “ng3” is farhigher than the possibility of other four classifications.

The classification information of the sample 440 is [0.5, 0.3, 0.1, 0.1,0.0]. Thus, there is possibly wrong classification if the sample 440 isclassified as the classification “OK”. That is because although thepossibility “0.5” of the classification “OK” is a little higher than thepossibility of other four classification, but not far higher than thepossibility of other four classification.

The classification information of the sample 450 is [0.3, 0.3, 0.1, 0.2,0.1]. Thus, there is possibly wrong classification if the sample 450 isclassified as the classification “OK” or “ng1”. That is because althoughthe possibility “0.3” of the classification “OK” or “ng1” is a littlehigher than the possibility of other four classification, but not farhigher than the possibility of other four classification.

Thus, as shown in FIG. 4A, in prior art, the sample 440 may beclassified as “OK” and the sample 450 may be classified as “OK” (or“ng1”). But, there is possibly wrong classification (or said, ambiguousdefect).

However, in an embodiment of the application, if the AI training module150 performs the classification flow of FIG. 3 on the samples (forexample, the samples 440 or 450) having high error-classificationpossibility, the classification results of the samples 440 or 450 may beshown in FIG. 5A or FIG. 5B.

That is, as shown in FIG. 5A, the classification information of thesample 440 is [0.5, 0.3, 0.1, 0.1, 0.0]. After similarity measurement,the five similarity distance is Δd1 (or a1)=−9, Δd2 (or a2)=1, Δd3 (ora3)=5, Δd4 (or a4)=3, and Δd5 (or a5)=5. Δd1 (or a1) refers thesimilarity distance between the classification information of the sample440 and the ideal classification information “OK”; Δd2 (or a2) refersthe similarity distance between the classification information of thesample 440 and the ideal classification information “ng1”; Δd3 (or a3)refers the similarity distance between the classification information ofthe sample 440 and the ideal classification information “ng2”; Δd4 (ora4) refers the similarity distance between the classificationinformation of the sample 440 and the ideal classification information“ng3”; and Δd5 (or a5) refers the similarity distance between theclassification information of the sample 440 and the idealclassification information “ng4”.

Weighting analysis is performed and accordingly the classificationinformation having high similarity is assigned by heavy weighting. Allsimilarity distances are added (Δd1+ . . . +Δd5) and the respectivesimilarity distance is divided by the summation of the similaritydistances to obtain the following values: Δd1/(Δd1+ . . . +Δd5)=(−9/5),Δd2/(Δd1+ . . . +Δd5)=(1/5), Δd3/(Δd1+ . . . +Δd5)=(5/5), Δd4/(Δd1+ . .. +Δd5)=(3/5), and Δd5/(Δd1+ . . . +Δd5)=(5/5). Then, normalizationcalculation is performed to obtain the weighted classificationinformation (w=n−1=5−1=4).

$\frac{{\sum\limits_{1}^{n}{ai}} - {a\; 1}}{w \times {\sum\limits_{1}^{n}{ai}}} = {{\left( {{\sum\limits_{1}^{n}{ai}} - {a\; 1}} \right)/\left\lbrack {w*\left( {{a\; 1} + \ldots + {a\; 5}} \right)} \right\rbrack} = \left\lbrack {{\left( {5 - \left( {- 9} \right)} \right\rbrack/\left( {4*5} \right)} = {{0.7\frac{{\sum\limits_{1}^{n}{ai}} - {a\; 2}}{w \times {\sum\limits_{1}^{n}{ai}}}} = {{\left( {{\sum\limits_{1}^{n}{ai}} - {a\; 2}} \right)/\left\lbrack {w*\left( {{a\; 1} + \ldots + {a\; 5}} \right)} \right\rbrack} = {{\left( {5 - 1} \right)/\left( {4*5} \right)} = {{0.2\frac{{\sum\limits_{1}^{n}{ai}} - {a\; 3}}{w \times {\sum\limits_{1}^{n}{ai}}}} = {{\left( {{\sum\limits_{1}^{n}{ai}} - {a\; 3}} \right)/\left\lbrack {w*\left( {{a\; 1} + \ldots + {a\; 5}} \right)} \right\rbrack} = {{\left( {5 - 5} \right)/\left( {4*5} \right)} = {{0\frac{{\sum\limits_{1}^{n}{ai}} - {a\; 4}}{w \times {\sum\limits_{1}^{n}{ai}}}} = {{\left( {{\sum\limits_{1}^{n}{ai}} - {a\; 4}} \right)/\left\lbrack {w*\left( {{a\; 1} + \ldots + {a\; 5}} \right)} \right\rbrack} = {{\left( {5 - 3} \right)/\left( {4*5} \right)} = {{0.1\frac{{\sum\limits_{1}^{n}{ai}} - {a\; 5}}{w \times {\sum\limits_{1}^{n}{ai}}}} = {{\left( {{\sum\limits_{1}^{n}{ai}} - {a\; 5}} \right)/\left\lbrack {w*\left( {{a\; 1} + \ldots + {a\; 5}} \right)} \right\rbrack} = {{\left( {5 - 5} \right)/\left( {4*5} \right)} = 0}}}}}}}}}}}}} \right.}$

Thus, from the normalization weighting analysis, the sample 440 shouldbe determined as “OK” classification, and the determination has lowoverkill possibility.

Similarly, as shown in FIG. 5B, the classification information of thesample 450 is [0.3, 0.3, 0.1, 0.2, 0.1]. After similarity measurement,the five similarity distance is Δd1 (or a1)=18, Δd2 (or a2)=−42, Δd3 (ora3)=18, Δd4 (or a4)=18, and Δd5 (or a5)=18. Δd1 (or a1) refers thesimilarity distance between the classification information of the sample450 and the ideal classification information “OK”; Δd2 (or a2) refersthe similarity distance between the classification information of thesample 450 and the ideal classification information “ng1”; Δd3 (or a3)refers the similarity distance between the classification information ofthe sample 450 and the ideal classification information “ng2”; Δd4 (ora4) refers the similarity distance between the classificationinformation of the sample 450 and the ideal classification information“ng3”; and Δd5 (or a5) refers the similarity distance between theclassification information of the sample 450 and the idealclassification information “ng4”.

Weighting analysis is performed and accordingly the classificationinformation having high similarity is assigned by heavy weighting. Allsimilarity distances are added (Δd1+ . . . +Δd5) and the respectivesimilarity distance is divided by the summation of the similaritydistances to obtain the following values: Δd1/(Δd1+ . . . +Δd5)=(18/30),Δd2/(Δd1+ . . . +Δd5)=(−42/30), Δd3/(Δd1+ . . . +Δd5)=(18/30), Δd4/(Δd1+. . . +Δd5)=(18/30), and Δd5/(Δd1+ . . . +Δd5)=(18/30). Then,normalization calculation is performed to obtain the weightedclassification information (w=n−1=5−1=4).

$\frac{{\sum\limits_{1}^{n}{ai}} - {a\; 1}}{w \times {\sum\limits_{1}^{n}{ai}}} = {{\left( {{\sum\limits_{1}^{n}{ai}} - {a\; 1}} \right)/\left\lbrack {w*\left( {{a\; 1} + \ldots + {a\; 5}} \right)} \right\rbrack} = {{\left( {30 - 18} \right)/\left( {4*30} \right)} = 0.1}}$$\frac{{\sum\limits_{1}^{n}{ai}} - {a\; 2}}{w \times {\sum\limits_{1}^{n}{ai}}} = {{\left( {{\sum\limits_{1}^{n}{ai}} - {a\; 2}} \right)/\left\lbrack {w*\left( {{a\; 1} + \ldots + {a\; 5}} \right)} \right\rbrack} = {{\left\lbrack {30 - \left( {- 42} \right)} \right\rbrack/\left( {4*30} \right)} = {{0.6\frac{{\sum\limits_{1}^{n}{ai}} - {a\; 3}}{w \times {\sum\limits_{1}^{n}{ai}}}} = {{{\left( {{\sum\limits_{1}^{n}{ai}} - {a\; 3}} \right)/\left\lbrack {w*\left( {{a\; 1} + \ldots + {a\; 5}} \right)} \right\rbrack}{\left( {30 - 18} \right)/\left( {4*30} \right)}} = {{0.1\frac{{\sum\limits_{1}^{n}{ai}} - {a\; 4}}{w \times {\sum\limits_{1}^{n}{ai}}}} = {{\left( {{\sum\limits_{1}^{n}{ai}} - {a\; 4}} \right)/\left\lbrack {w*\left( {{a\; 1} + \ldots + {a\; 5}} \right)} \right\rbrack} = {{\left( {30 - 18} \right)/\left( {4*30} \right)} = {{0.1\frac{{\sum\limits_{1}^{n}{ai}} - {a\; 5}}{w \times {\sum\limits_{1}^{n}{ai}}}} = {{\left( {{\sum\limits_{1}^{n}{ai}} - {a\; 5}} \right)/\left\lbrack {w*\left( {{a\; 1} + \ldots + {a\; 5}} \right)} \right\rbrack} = {{\left( {30 - 18} \right)/\left( {4*30} \right)} = 0.1}}}}}}}}}}$

Thus, from the normalization weighting analysis, the sample 450 shouldbe determined as “ng1” classification, and the determination has lowoverkill possibility.

By comparison of FIG. 4A, FIG. 4B, FIG. 5A and FIG. 5B, in an embodimentof the application, after performing kernel function on the sampleswhich may be possibly wrong classified, the classification informationhaving ambiguous defect may higher possibility than other classificationinformation.

In an embodiment of the application, machine learning is used to improvethe detection rate of the AOI device.

An embodiment of the application combines the machine visual technologyand Maximum Entropy Convolutional Neural Network (MECNN) to lower theleakage rate and the overkill rate, and the normal samples areclassified by performing characteristic weight analysis on theoverkilled defects. Further, recursive deep learning is achieved (forexample, the classification result of the step 240 of FIG. 2 may be fedinto the AI training module 150 for training the AI training module 150again), to lower the overkill rate. Besides, the AI training module 150may be operated along with the existing AOI device or AOI software. Or,the AI training module 150 may be operated independently. Thus, theflexibility is improved.

In an embodiment of the application, the discrete output results areanalyzed by weighting characteristic, for effectively classify thesamples having ambiguous defect (which is easily occurred in existingAOI device). Further, the new classification results may be recursivelyfed back to the AI training module, to enhance the robustness of the AItraining module and to improve detection rate of AOI device.

Further, an embodiment of the application provides a computer-readablemedia. When the computer-readable media is loaded by an AOI imageclassification system, the AOI image classification system may executethe AOI image classification method in FIG. 2, FIG. 3A, FIG. 3B, FIG. 5Bor FIG. 5B.

It will be apparent to those skilled in the art that variousmodifications and variations can be made to the disclosed embodiments.It is intended that the specification and examples be considered asexemplary only, with a true scope of the disclosure being indicated bythe following claims and their equivalents.

What is claimed is:
 1. An automated optical inspection (AOI) imageclassification method including: feeding a plurality of NG informationof a plurality of samples from an AOI device into an artificialintelligence (AI) training module; performing discrete output on theplurality of NG information of the samples by the AI training module togenerate a plurality of classification information of the samples;performing kernel function on the plurality of classificationinformation of the samples by the AI training module to calculaterespective similarity distances of the samples and to perform weightinganalysis; performing classification determination based on weightanalysis results of the samples, to determine respective classificationresults of the samples; and based on the respective classificationresults of the samples, classifying the samples; wherein the step ofcalculating respective similarity distances of the samples includes:calculating a plurality of similarity distances between theclassification information of the samples and a plurality of idealclassification.
 2. The method according to claim 1, wherein the step ofperform weighting analysis includes: assigning respective weighting tothe similarity distances of the samples, wherein if a first similaritydistance of a first sample is smaller than a second similarity distanceof the first sample, a first weight assigned to the first similaritydistance of the first sample is higher than a second weight assigned tothe second similarity distance of the first sample.
 3. The methodaccording to claim 1, wherein the classification results of the samplesare fed back to the AI training module for training the AI trainingmodule.
 4. The method according to claim 1, wherein the step ofclassifying the samples includes: classifying by a classification deviceincluding any combination of a pneumatic cylinder, a conveyor, acantilever mechanism, a robot arm, and a carrier.
 5. A non-transitorycomputer-readable media, when loaded by an automated optical inspection(AOI) image classification system, the AOI image classification systemexecuting the AOI image classification method in claim
 1. 6. Anautomated optical inspection (AOI) image classification systemincluding: an AOI device, for performing automated optical inspection ona plurality of samples to obtain respective OK information or NGinformation of the samples; and an artificial intelligence (AI) trainingmodule coupled to the AOI device, the AI training module receiving theplurality of NG information of the samples from the AOI device, the AItraining module performing discrete output on the NG information of thesamples to generate a plurality of classification information of thesamples, the AI training module performing kernel function on theplurality of classification information of the samples to calculaterespective similarity distances of the samples and to perform weightinganalysis, and the AI training module performing classificationdetermination based on weight analysis results of the samples, todetermine respective classification results of the samples; wherein theAI training module calculates a plurality of similarity distancesbetween the classification information of the samples and a plurality ofideal classification.
 7. The system according to claim 1, wherein the AItraining module assigns respective weighting to the similarity distancesof the samples, wherein if a first similarity distance of a first sampleis smaller than a second similarity distance of the first sample, afirst weight assigned to the first similarity distance of the firstsample is higher than a second weight assigned to the second similaritydistance of the first sample.
 8. The system according to claim 6,wherein the classification results of the samples are fed back to the AItraining module for training the AI training module.
 9. The systemaccording to claim 6, wherein the classification results, obtained bythe AI training module, of the samples are classified by aclassification device including any combination of a pneumatic cylinder,a conveyor, a cantilever mechanism, a robot arm, and a carrier.